Where the training of machine learning models is concerned, there's always a risk of overfitting -- or corresponding to closely -- to a particular set of data. In point of fact, it's not infeasible that popular machine learning benchmarks like the Arcade Learning Environment encourage overfitting, in that they have a low emphasis on generalization. That's why OpenAI -- the San Francisco-based research firm cofounded by CTO Greg Brockman, chief scientist Ilya Sutskever, and others -- today released the Procgen Benchmark, a set of 16 procedurally-generated environments (CoinRun, StarPilot, CaveFlyer, Dodgeball, FruitBot, Chaser, Miner, Jumper, Leaper, Maze, BigFish, Heist, Climber, Plunder, Ninja, and BossFight) that measure how quickly a model learns generalizable skills. It builds atop the startup's CoinRun toolset, which used procedural generation to construct sets of training and test levels. "We want the best of both worlds: a benchmark comprised of many diverse environments, each of which fundamentally requires generalization," wrote OpenAI in a blog post.